a head location/posture estimation step of estimating a location/posture of a 
viewpoint of a player using the detected marker positions; 

a mixed reahty space linage generation step of generating a virtual space 
image to be observed by the player Based on the location/posture of the viewpoint of the 
^* player and model information; and 

an image combining step ot generating a mixed reality space image by 
combining the real space image with the Virtual space image, 

wherein the markers are laid oat in the real space so that the markers to be 
used by only a given player are laid out at positions hidden by real objects when the 
markers are observed from the other player,— \ 

REMARKS 

Claims 1-3, 5-8 and 15-21 are pending in this application, with Claims 1,15, 
16, 18, 19, 20 and 21 being independent. Claims 4 and 9-14 have been cancelled without 
prejudice. 

Claims 1, 5 and 15-19 have been amended. Applicants submit that support for 
the amendments can be found in the original disclosure, and therefore no new matter has 
been added. 

Claims 4, 5, 8 and 15-17 are objected to. Although the Office Action does not 
state the basis for the objection. Applicants understand the objection to be that the claims 
contain allowable subject matter but are in improper form because they depend from a 
base claim. 





Claims 1-3, 6-7, 10-11 and 13 are rejected under 35 U.S.C. § 102(b) as being 
anticipated by U.S. Patent No. 6,054,991 (Crane, et al.). 

Claims 10, 1 1 and 13 also appear to be rejected as being anticipated by U.S. 
Patent No. 6,094,625 (Ralston). 

Claim 9 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Crane. 

et al. 

Claim 12 is rejected under 35 U.S.C. § 103(a) as being unpatentable over 

Ralston . 

Claims 14 and 18-19 are rejected under 35 U.S.C. § 103(a) as being 
unpatentable over U.S. Patent No. 6,064,749 (Hirota, et al.) in view of Crane, et al. 
The rejections respectfully are traversed. 

In one aspect of the invention as recited in Claim 1, a marker layout method 
for laying out markers in a real space as position indices upon presenting a mixed reality 
space comprises the step of laying out the markers to have a positional relationship that 
allows a given player not to observe markers to be used by only another player when a 
plurality of players who observe the mixed reality space within different movable ranges 
observe the mixed reality space. The markers to be used by only the given player are laid 
out at positions hidden by real objects when the markers are observed from the other 
player. 

In another aspect, the invention is directed to a mixed reality space image 
generation method for generating a mixed reality space image which makes a player 
experience mixed reality by mixing a real space in which markers serving as position 
indices are laid out, and a virtual space. As recited in Claims 15 and 16, the method 
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comprises a marker detection step of detecting the markers from image data obtained by 
sensing the real space from a substantially viewpoint position of the player, a correcting 
step of correcting location/posture information of the player based on the markers detected 
by the marker detection step, and a mixed reality space image generation step of 
generating a mixed reality space image to be observed by the player, so the player observes 
virtual object images that do not include any images of the markers in marker regions 
including the markers from the image data. 

As recited in Claim 15, the mixed reality space image generation step includes 
a step of substituting or overlaying images of the marker regions by predetermined virtual 
object images. 

As recited in Claim 16, the mixed reality space image generation step includes 
a step of substituting or overiaying all the markers in the image data with predetermined 
virtual object images. 

Claims 18 and 19 are directed to computer readable storage media and 
correspond substantially to Claims 15 and 16. 

Newly-presented Claims 20 and 21 are directed to a mixed reality apparatus 
and a method for providing a mixed reality space shared by a plurality of players and each 
recites, among other things, inputting a real space image representing the real space, 
extracting pixels having a color that is predefined as a marker from the real space image, 
performing a labeling process to detect marker regions, and detecting positions of markers 
in the marker regions. Markers are laid out in the real space so that the markers to be used 
by only a given player are laid out at positions hidden by real objects when the markers are 
observed from another player. 
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Preliminary to discussing the relevant art, Applicants wish to point out that 
Claim 1 has been amended to include the features of allowable Claim 4, and allowable 
Claims 15 and 16 have been amended to be in independent form. Accordingly, Applicants 
submit that Claims 1-3, 5-8, 15 and 16 are allowable. 

Turning to the rejection of the remaining independent claims, i.e., Claims 18 
and 19, Applicants submit that Hirota, et al. discloses using landmarks laid out in the real 
space to enhance accuracy of registration of computer-generated images with real world 
objects. 

Applicants submit that Crane, et al. discloses a virtual reality system 20 in 
which an operator perceives being in a virtual reality world 74 via a video display 60 in a 
helmet 26. 

However, Applicants submit that neither Hirota, et al. nor Crane, et al. . even in 
combination, discloses or suggests, among other things, substituting or overlaying images 
of the marker regions by predetermined virtual object images, as recited in Claim 18, or 
substituting or overlaying an image of a region including all the markers in the image data 
by predetermined virtual object images, as recited in Claim 19. What is more, as noted 
above. Claims 18 and 19 correspond to Claims 15 and 16, respectively, which should now 
be allowable. 

Accordingly, withdrawal of the rejection of Claims 18 and 19 respectfully is 

requested. 

Newly-submitted Claims 20 and 21 are also submitted to be allowable because 
none of the applied art discloses or suggests, among other things, markers laid out in the 
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real space so that the markers to be used by only the given player are laid out at positions 
hidden by real objects when the markers are observed from another player. 



For the foregoing reasons, Applicants submit that this application is in 



condition for allowance. Favorable reconsideration, withdrawal of the rejections and 
objections set forth in the above-mentioned Office Action and an early Notice of 
Allowance are requested. 



Applicants' undersigned attorney may be reached in our Washington, D.C. 



office by telephone at (202) 530-1010. All correspondence should continue to be directed 
to our below-listed address. 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3801 
Facsimile: (212)218-2200 

CAW:a)T 



Respectfully submitted, 




Attorney for Applicants 
Collin A. Webb 
Registration No. 44,396 
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APPENDIX 



MARKED-UP VERSION SHOWING AMENDMENTS TO CLAIMS 

1. (Amended) A marker layout method for laying out markers in a real space as 
position indices upon presenting a mixed reality space, comprising the step of: 

laying out the markers to have a positional relationship that allows a given player 
not to observe markers to be used by only another player when a plurality of players who 
observe the mixed reality space within different movable ranges observe the mixed reality 
space^ 

wherein the markers to be used bv onlv the given player are laid out at positions 
hidden by real objects when the markers are observed from the other player . 

4. (Cancelled) 

5. (Amended) The method according to claim [4] 1, wherein the real objects are 
laid out for an application that uses the mixed reality space. 

9 through 14. (Cancelled) 

15. (Amended) [The method according to claim 14,] A mixed reality space image 
generation method for generating a mixed reality space image which makes a player 
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experience mixed reality by mixing a real space in which markers serving as position 
indices are laid out, and a yirtual space, comprising: 

a marker detection step of detecting the markers from image data obtained by 
sensing the real space from a substantially viewpoint position of the player: 

a correcting step of correcting location/posture information of the player based on 
the markers detected by said marker detection step: and 

a mixed reality space image generation step of generating a mixed reality space 
image to be observed by the player, so the player observes virtual object images that do not 
include any images of the markers in marker regions including the markers from the image 
data. 

wherein [the] said mixed reality space image generation step includes [the] a step 
of substituting or overlaying images of the marker regions by predetermined virtual object 
images. 

16. (Amended) [The method according to claim 14,] A mixed reality space image 
generation method for generating a mixed reality space image which makes a player 
experience mixed reality by mixing a real space in which markers serving as position 
indices are laid out, and a virtual space, comprising: 

a marker detection step of detecting the markers from image data obtained by 
sensing the real space from a substantially viewpoint position of the player: 
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a correcting step of correcting location/posture information of the plaver based on 
the markers detected by said marker detection step: and 

a mixed reality space image generation step of generating a mixed reality space 
image to be observed bv the plaver. so the plaver observes virtual object images that do not 
include any images of the markers in marker regions including the markers from the image 
data. 

wherein [the] said mixed reality space image generation step includes [the] a step 
of substituting or overlaying [an image of a region including] all the markers in the image 
data [by a] with predetermined virtual object [image] images . 

17. (Amended) The method according to claim 15, wherein the predetermined 
virtual [objects are] object images are images of plane patches with the same or similar 
texture, size, location and posture of [the] respective marker regions before the markers are 
laid out. 

18. (Amended) A computer readable storage medium that stores a program which 
can be executed by a computer, said program making the computer perform the steps of 
[which executes the program function as]: 

[a mixed reality apparatus for making a player experience mixed reality by making 
the player observe a mixed reality space image obtained by mixing a real space in which 
markers serving as position indices are laid out, and a virtual space, comprising:] 
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a marker detection [means for] step of detecting [the] markers , serving as position 
indices, from image data obtained by sensing [the] a real space from a substantially 
viewpoint position of [the] a player; [and] 

a correcting step of correcting location/posture information of the player based on 
the markers detected by said marker detection step: and 

a mixed reality space image generation [means for] step of generating [the] a mixed 
reality space image to be observed by the player, so the player observes virtual object 
images that do not include any images of the markers in [surrounding] marker regions 
[(marker regions)] including the markers [in] from the image data^ 

wherein said mixed reality space image generation step includes substituting or 
overlaying images of the marker regions by predetermined virtual object images . 

19. (Amended) A computer readable storage medium [which] that stores [a mixed 
reality space image generation program for generating a mixed reality space image which 
makes a player experience mixed reality by mixing a real space in which markers serving 
as position indices are laid out, and a virtual space, storing] a program which can be 
executed by a computer, said program making the computer perform the steps of : 

a correcting step of correcting location/posture information of the player based on 
the markers detected by said marker detection step: and 

a marker detection [program] step of detecting the markers , serving as position 
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indices, from image data obtained by sensing [the] a real space from a substantially 
viewpoint position of [the] a player; [and] 

a mixed reality space image generation [program] step of generating [the] a mixed 
reality space image to be observed by the player, so the player observes virtual object 
images that do not include any images of the markers in [surrounding] marker regions 
[(marker regions)] including the markers [in] from the image data^ 

wherein said mixed reality space image generation step includes substituting or 
overlaying an image of a region including all the markers in the image data by 
predetermined virtual object images . 
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